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screen  for  subjects  involved  in  ATCS-related  research. 
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Comparison  of  Performance  on  the  Shipley  Institute  of  Living  Scale, 
Air  Traffic  Control  Specialist  Selection  Test  and 
FAA  Academy  Screen 


Current  research  to  evaluate  the  effects  of  stressors  for 
Air  Traffic  Control  Specialists  (ATCSs)  on  performance, 
physiological,  biochemical,  and  subjective  responses  re¬ 
quires  use  of  non-ATCS  subjects  in  the  laboratory.  This 
prompted  efforts  to  find  screening  measures  for  selecting 
non-ATCS  research  subjects  with  characteristics  similar 
to  the  ATCS  population.  The  present  study  was  con¬ 
ducted  to  establish  normative  data  for  ATCSs  on  a 
measure  of  general  intelligence  using  the  Shipley  Insti¬ 
tute  of  Living  Scale  (SILS)  to  supplement  screening 
criteria  regarding  health,  educational  level,  and  work 
history. 

While  a  broad  body  of  literature  exists  on  selection  of 
ATCSs,  limited  data  have  been  reported  concerning  the 
average  IQ  of  this  population.  Brokaw  (1957)  reported 
data  from  the  California  Testing  Board  Test  of  Mental 
Maturity;  however,  he  only  reported  correlations  of  the 
sub-test  scores  with  other  aptitude  tests.  Karson  and 
O’Dell  reported  data  from  the  intelligence  factor  on  the 
16  Personality  FactorTest  (16PF)  for  20,933  ATCSs  in 
1 974.  This  ATCS  group  was  found  to  have  scores  on  this 
measure  of  intelligence  nearly  one  standard  deviation 
above  the  general  population,  or  comparable  to  a  Wechsler 
IQ  score  of  about  115.  The  16PF  scores  on  the  intelli¬ 
gence  factor  (B)  from  a  more  recent  ATCS  applicant 
group  (Schroeder  and  Dollar,  1989)  were  identical  to 
those  of  Karson  and  O’Dell  (1974). 

For  purposes  of  this  study,  a  relatively  short,  easily 
administered  global  measure  of  intelligence  was  required. 
The  Shipley  Institute  of  Living  Scale  (SILS)  was  selected 
because  it  1)  provides  both  verbal  (vocabulary)  and 
cognitive  performance  (abstraction)  measures  of  general 
intelligence  functioning;  2)  can  be  administered  in  a 
short  amount  of  time  to  a  large  group;  3)  provides 
estimated  Wechsler  Adult  Intelligence  Scale-Revised 
(WAIS-R)  Full  Scale  Intelligence  scores;  4)  has  reason¬ 
able  test-retest  reliabilities  and  validity  (Martin, 
Friedmeyer,  Sterne,  and  Brittain,  1977;  Goodman, 
Streiner,  and  Woodward,  1 974;  and  Zachary,  1 986);  5) 
has  been  widely  used  (Martin,  Blair,  and  Vickers,  1979; 
Young  and  Rearden,  1979;  and  Guilberstadt,  Lushene 


and  Buegel,  1976);  and  6)  has  been  used  previously  for 
subject  selection  in  aviation-related  research  (Mertens 
and  Collins,  1 986).  However,  no  data  had  been  collected 
on  the  ATCS  population. 

Normative  data  on  the  SILS  was  collected  on  a  sample 
of  ATCS  students  entering  the  FAA  Academy  for  future 
reference  in  selecting  non-ATCS  research  subjects  for  air 
traffic  control  related  research.  In  addition,  this  provided 
an  opportunity  to  investigate  the  relationship  between 
general  intelligence,  as  measured  by  the  SILS,  and  ATCS 
selection  measures  which  include  the  Office  of  Personnel 
Management  (OPM)  Air  Traffic  Control  test  battery 
and  the  FAA  Academy  nonradar  screen  program  (NSP). 

METHOD 


Subjects 

Three  groups  of  students  (N=563)  entering  the  FAA 
Academy  in  September  1 990  and  in  August  and  October 
1991  were  used  for  this  study.  At  that  time,  ATCS 
applicants  underwent  a  two  stage  selection  process,  (a) 
administration  of  the  Office  of  Personnel  Management 
(OPM)  Air  Traffic  Control  Specialist  Test  Battery  and 
(b)  participation  in  the  nonradar  screen  program  (NSP), 
a  nine  week  performance-based  screening  course  at  the 
FAA  Academy  in  Oklahoma  City.  Students  from  this 
sample  who  withdrew  prior  to  completing  the  NSP, 
those  who  had  previously  participated  in  the  NSP,  and 
those  for  whom  OPM  data  were  not  available,  were 
excluded  from  the  sample.  There  were  414  of  the  total 
563  students  who  were  included  in  the  final  sample. 
Twenty-nine  percent  (121)  of  the  final  sample  were 
women  and  79%  (328)  were  non-minorities.  Fifty-nine 
percent  (243)  of  the  final  sample  reported  having 
completed  some  college,  while  31%  (128)  reported 
completing  a  college  degree.  The  remaining  10%  (43) 
completed  high  school.  The  average  age  of  this  final 
sample  was  26. 1  (age  range  from  1 9  to  34).  All  students 
in  the  final  sample  had  successfully  completed  the  first 


stage  of  the  selection  process;  that  is,  they  had  received  a 


score  of  70  or  better  on  the  OPM  battery. 
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Procedure 

The  SILS  (1939)  was  administered  to  students  upon 
their  arrival  at  the  FAA  Academy  as  part  of  a  three  hour 
experimental  testing  session  during  orientation. 
Administration  of  the  test  was  conducted  according  to 
the  protocol  in  the  SILS  manual  (Zachary,  1986). 
Students  were  allowed  the  designated  10  minutes  for 
each  of  the  two  SILS  sections,  Vocabulary  and 
Abstraction.  Participation  was  voluntary  and  did  not 
influence  their  Academy  scores.  The  test  was  administered 
immediately  following  a  1 5  minute  break.  Following  the 
orientation  period,  students  proceeded  into  the  nine 
week  nonradar  screen  program.  Biographical,  OPM, 
and  Academy  NSP  performance  scores  were  obtained 
after  students  completed  the  NSP  from  a  data  base 
maintained  by  the  Selection  and  Validation  Research 
Section  at  the  Civil  Aeromedical  Institute. 

Measures 

The  SILS  yields  two  raw  scores.  Vocabulary  and 
Abstraction,  and  a  total  score  resulting  from  adding  the 
weighted  scores  in  which  Abstraction  receives  a  weight  of 
2.  SILS  total  scores  were  converted  to  estimated  WAIS- 
R  scores  in  accordance  with  the  age-referenced  conversion 
tables  in  the  SILS  manual. 

The  Office  of  Personnel  Management  (OPM)  Air 
T  raffle  Control  rating  consists  of  a  composite  score  from 
the  ATC  test  battery  plus  applicable  veteran’s  points. 
The  ATCS  test  battery  is  comprised  of  an  abstract 
reasoning  test  (ABSR),  as  well  as  the  Multiplex  Control¬ 
ler  Aptitude  Test  (MCAT).  These  are  described  in  more 
detail  elsewhere  (Manning,  Kegg,  and  Collins,  1988). 
The  MCAT  receives  a  weight  of  4,  while  the  Abstract 
Reasoning  receives  a  weight  of  1  in  the  composite 
calculations.  The  Occupational  Knowledge  Test  (OKT) 
assesses  the  applicant’s  knowledge  of  Air  T raffic  rules 
and  regulations.  Points,  up  to  1 5,  are  added  to  the  OPM 
rating  for  scores  on  the  OKT  over  50.  Veteran’s  points 
are  added  to  the  composite. 

The  nine  week  Academy  NSP  was  composed  of  two 
components;  academics  and  laboratory  (Della  Rocco, 
Manning,  and  Wing,  1991).  Thirteen  performance 
measures  were  collected  on  each  student  during  the 
NSP.  The  NSP  final  composite  score  (COMP)  is  com¬ 
prised  of  twelve  component  scores  as  described  below. 


In  the  academic  component,  students  were  taught  air 
traffic  control  (ATC)  rules  and  procedures.  Six  perfor¬ 
mance  measures  were  taken  from  multiple  choice  aca¬ 
demic  exams  and  a  map  test  of  the  synthetic  airspace. 
These  received  a  weight  in  the  composite  of  20%.  Scores 
from  four  multiple-choice  tests  and  the  map  test  were 
combined  into  a  Block  Test  Average  (BA).  The  final 
multiple-choice  exam  was  the  Comprehensive  Course 
Test  (CCT). 

In  the  laboratory  component,  students  were  asked  to 
demonstrate  application  of  ATC  rules  and  procedures  in 
scripted  30  minute  scenarios.  Six  standardized  labora¬ 
tory  problems  were  formally  graded.  A  trudent  received 
a  composite  of  two  scores  from  a  grading  instructor  on 
each  problem.  The  first  score,  the  technical  assessment 
(TA),  was  a  numeric  assessment  of  the  student’s  errors  in 
application  of  ATC  rules  and  procedures.  The  second 
score,  the  instructor  assessment  (lA),  was  a  subjective 
instructor  rating  of  the  student’s  global  performance  on 
the  problem.  The  student’s  lowest  of  the  six  graded 
laboratory  problems  was  dropped  from  the  final  compre¬ 
hensive  NSP  score.  Thus,  five  of  the  scores  were  from 
graded  laboratory  problems  and  received  a  weight  of 
60%  in  the  final  score.  These  are  reported  for  this  study 
as  the  average  of  the  five  problems  (AVL5).  Finally,  a 
timed  multiple  choice  final  exam,  the  Controller  Skills 
Test  (CST),  comprised  20%  of  the  final  NSP  score. 

Students  with  NSP  final  composite  scores  of  70  or 
greater  passed  and  proceeded  into  air  traffic  training.  A 
final  composite  score  of  less  than  70  resulted  in  failure 
and  separation  from  the  occupation. 

Analyses 

Two  analyses  were  conducted  in  this  study.  The  first 
analysis  was  designed  to  determine  whether  or  not 
successful  students  achieved  significantly  higher  scores 
on  the  SILS  than  unsuccessful  students.  A  multivariate 
analysis  ofvariance  (MANOVA)  was  utilized  to  compare 
mean  SILS  scores.  The  second  analysis  was  conducted  to 
determine  the  correlations  between  SILS  scores  and 
OPM  and  NSP  scores.  Because  students  entering  the 
Academy  had  already  been  screened  by  the  OPM  Air 
Traffic  Test  Battery,  it  was  important  to  correct  for  the 
restriction  in  the  range  of  SILS  scores  caused  by  selection 
based  on  the  OPM  battery  (Boone  and  Lewis,  1978; 
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Thorndike,  1949).  To  correct  for  restriction  in  range, 
population  OPM  scores  and  variance  were  calculated 
from  data  supplied  to  the  Civil  Aeromedical  Institute  by 
the  Office  of  Personnel  Management  for  persons  reporting 
adateofavailability  between  January  1988  and  December 
1989  (N=3,484). 

RESULTS 


Shipley  scores  for  the  sample  were  as  follows:  Vocabu¬ 
lary  3 1 .4  (sd=3.7);  Abstraction  35.7  {sd=3.5);  and  Total 
67.0  (sd=5.9). 

The  first  analysis  conducted  for  this  study  compared 
the  performance  of  those  who  passed  the  Academy  and 
those  who  failed.  Table  1  ptesents  descriptive  statistics 
for  scores  on  the  SILS  by  final  Academy  Status. 


Of  the  4l4  ATCS  students  in  final  sample,  233 
(56.3%)  passed  the  Academy  NSP,  181  (43.7%)  failed. 
The  average  estimated  WAIS-R  for  the  group  of  414 
Academy  entrants  was  109.8  (sd=5.9).  The  average 

TABLE  1.  Mean  Scores  on  the  SILS  by 
Academy  Status 


ACADEMY  STATUS 

Passed  Failed 

(N=233)  (N=181) 


SILS  SCORES 

AVE. 

SD 

AVE. 

SD 

Vocabulary  _ 

_31.9 

_ 3.8  _ 

_ 31.0  _ 

_3.5 

Abstraction  _ 

_36.2 

_ 3.1  _ 

_ 34.9  _ 

_3.9 

Total 

_68.0 

_ 5.8  _ 

_ 65.8  _ 

_5.8 

WAIS-R 

110.8 

_ 5.8 

_  108.6  ^ 

5.7 

(estimated) 

Results  of  a  Multivariate  Analysis  of  Variance  com¬ 
paring  SI  LS  scores  of  passing  students  to  those  who  failed 
were  significant  (F(4,409)=5.15,  p<.001).  Analysis  of 
Variance  for  each  measure  (df=l,4l2)  revealed  that 
students  who  passed  the  Academy  scored  significantly 
higher  on  all  SILS  measures  (Vocabulary  F=6.64,  p<.01 ; 
Abstraction  F=15.1 1,  p<.001:  Total  F=14.17,  p<.001; 
WA1S-RF=14.58,  p<.001). 

The  second  analysis  calculated  Pearson  correlations 
between  SILS  and  both  sets  of  ATCS  selection  mea¬ 
sures — OPM  and  NSP.  Table  2  presents  correlations 
between  OPM  and  SILS  measures.  Both  unadjusted 
correlations  and  correlations  adjusted  for  the  restriction 
in  range  due  to  prior  selection  of  subjects  by  the  OPM 
exam  arc  presented. 


TABLE  2.  Correlations  between  SILS  and  OPM  Scores 


Unadjusted  for  Restriction  in  Range  by  OPM 

SILS  Scores 


OPM  TEST  MEASURES 

Vocabulary 

Abstraction 

Total 

WAIS-R 

Multiplex  Controller  Aptitude  Test  (MCAT) 

^•13**  _ 

.10* 

_.14**  _ 

.13** 

Abstract  Reasoning  (ABSR) 

.09 

.18** 

.17** 

.17** 

Occupational  Knowledge  Test  (OKT) 

.08 

.01 

.05 

.03 

OPM  Rating 

.11* 

.08 

.12* 

.08 

Adjusted  for  Restriction  in  Range  by  OPM 

SILS  Scores 


OPM  TEST  MEASURES 

Vocabulary  Abstraction 

Total 

WAIS-R 

Multiplex  Controller  Aptitude  Test  (MCAT)  .33 

.26 

.36 

.33 

Abstract  Reasoning  (ABSR) 

.10 

.19 

.18 

.18 

Occupational  Knowledge  Test  (OKT) 

.07 

.01 

.04 

.03 

OPM  Rating 

.34 

.26 

.37 

.26 

•p<.05  **p<.01 
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TABLE  3.  Correlations  Between  SILS  and  NSP  Performance  Measures 


Unadjusted  for  Restriction  in 

Range  by  OPM 

SILS  Scores 

NSP 

Vocabulary  .Abstraction 

Total 

WAIS-R 

Block  Average  (BA) 

.14** 

.19** 

.20** 

.18** 

Comprehensive  Course  Test  (CCT) 

.16** 

_.18**  _ 

_ .21**  _ 

.18** 

Average  5  of  6  Labs  (AVL5) 

.09 

.23** 

.20** 

.20** 

Controller  Skills  Test  (CST) 

.21** 

.30** 

.30** 

.30** 

Final  Comprehensive  Score  (COMP) 

.14** 

_.27**  _ 

_ .24**  _ 

.24** 

*♦  p<.01 

Adjusted  for  Restriction  in 

Range  by  OPM 

SILS  Scores 

NSP 

Vocabulary  Abstraction 

Total 

WAIS-R 

Block  Average  (BA) 

.30 

.29 

.36 

.28 

Comprehensive  Course  Test  (CCT) 

.28 

.27 

_ .33  _ 

.27 

Average  5  of  6  Labs  (AVL5) 

.27 

.33 

.36 

.30 

Controller  Skills  Test  (CST) 

.35 

.38 

.43 

.38 

Final  Comprehensive  Score  (COMP) 

.34 

.38 

_ .43  _ 

.38 

Tlie  correlation  between  scores  from  SILS  Abstrac¬ 
tion  ard  the  OPM  Abstract  Reasoning,  although  statis¬ 
tically  significant,  was  notably  low  (.18),  suggesting  the 
possibility  of  differences  in  what  each  test  measures.  The 
highest  adjusted  correlations  were  found  between  all 
SILS  scores  and  MCAT  and  OPM  Rating. 

Table  3  presents  correlations  between  NSP  perfor¬ 
mance  measures  and  SILS  scores.  Correlations  were 
adjusted  for  restriction  in  range  of  SILS  scores  due  to 
prior  OPM  selection  utilizing  Thorndike’s  three  variable 
case  formula  (Thorndike,  1949). 

These  correlations  revealed  that  the  Abstraction  scores 
were  somewhat  better  than  the  Vocabulary  at  predicting 
NSP  performance.  This  was  particularly  true  for  the 
skill-based  performance  measures,  specifically  the  graded 
laboratory  problems  and  the  Controller  Skills  Test. 
Among  the  SILS  measures,  the  adjusted  SILS  Total  score 
was  the  best  predictor  of  NSP  performance.  Table  4 
presents  correlations  between  OPM  and  NSP  scores. 


Review  of  Table  4  reveals  that  correlations  between 
the  SILS  Abstraction  and  NSP  measures  from  Table  3 
were  notably  higher  than  the  correlations  between  OPM 
Abstract  Reasoning  (ABSR)  and  NSP  measures.  OPM 
Ratine  had  the  highest  adjusted  correlations  with  the 
skill-based  NSP  measures.  The  OKT  correlations  with 
NSP  performance  measures  are  unusually  high  in  this 
sample  when  compared  to  previous  reports  (Della  Rocco, 
et  al.,  1991). 

DISCUSSION 

The  purpose  of  this  study  was  to  administer  a  rela¬ 
tively  short,  global  measure  of  intelligence  to  ATCSs  in 
order  to  collect  normative  data  and  assess  the  relation¬ 
ship  between  SILS  measures  and  well-investigated  ATCS 
selection  measures.  Few  studies  had  reported  IQdata  for 
this  population  and  none  had  been  reported  for  the  SILS. 

Data  were  collected  from  a  sample  of  414  ATCSs 
prior  to  entering  the  second  stage  of  the  ATCS  selection 
process,  the  Nonradar  Screen  Program.  Of  the  people 
who  passed,  the  average  estimated  WAIS-R  scores  were 
found  to  be  1 10.8  (sd=6).Thus,  as  a  group,  ATCSs  tend 
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TABLE  4.  Correlations  between  OPM  Scores  and  NSP  Performance  Measures 


Unadjusted  for  Restriction  in  Range  by  OPM 

NSP 

MCAT 

OPM  Scores 

ABSR  OKT 

Rating 

Block  Average  (BA) 

.11* 

.12* 

.11** 

.22** 

Comprehensive  Course  Test  (CCT) 

.10* 

.04 

k) 

* 

* 

1 

_ .16** 

Average  5  of  6  Labs  (AVL5) 

.16** 

.14** 

.25** 

.24** 

Controller  Skills  Test  (CST) 

.19** 

.10* 

.21** 

.22** 

Final  Comprehensive  Score 

.18** 

.14** 

.28** 

.25** 

**  p<.01 

Adjusted  for  Restriction  in 

Range  by  OPM 

NSP 

MCAT 

OPM  Scores 

ABSR  OKT 

Rating 

MCAT  ABSR  OKT  Rating 

Block  Average  (BA) 

.29 

.1.1 

.28 

.60 

Comprehensive  Course  Test  (CCT) 

.26 

.04 

.26  _ 

_ .47 

Average  5  of  6  Labs  (AVL5) 

.40 

.15 

.22 

.63 

Controller  Skills  Test  (CST) 

.46 

.1 1 

.21 

.60 

Final  Comprehensive  Score 

.44 

.15 

.25 

.65 

to  be  above  average  in  intelligence.  This  finding  is  similar 
to  the  findings  of  Karson  and  O’Dell  (1974).  The 
students  who  passed  the  NSP  were  found  to  have  scored 
significantly  higher  than  unsuccessful  students  on  each 
of  the  SILS  measures.  Correlations  between  the  SILS  and 
ATCS  selection  measures,  OPM  and  NSP  scores  respec¬ 
tively,  were  found  to  be  low  to  moderate,  but  similar  to 
correlations  found  in  early  work  on  ATCS  selection 
(Brokaw,  1984).  Because  of  the  research  leading  to  the 
use  of  an  abstract  reasoning  test  in  the  OPM  battery 
(Collins,  Boone,  and  VanDeventer,  1984),  it  was  antici¬ 
pated  that  the  Abstraction  would  be  a  better  predictor  of 
NSP  performance  than  the  Vocabulary.  This  was  found 
to  be  the  case. 

Specifically,  the  Abstraction  scores  were  found  to 
correlate  better  than  Vocabulary  scores  with  the  perfor¬ 
mance-based  NSP  measures  (i.e.,  the  laboratory  grades 
and  the  Controller  Skills  Test).  Comparison  of  the 
adjusted  correlations  from  OPM  and  NSP  scores  in 
Table  4  with  those  of  the  SILS  and  NSP  scores  from 
Table  3  suggests  that  the  MCAT  and  SILS  Total  were 
comparable  predictors  of  NSP  performance.  The  OPM 
Rating  was  the  best  predictor  of  NSP  performance. 
Although  MCAT  and  OPM  Rating  would  be  desirable 
for  use  in  selection  of  subjects  in  our  ATCS  stressor 


studies,  they  are  controlled  tests  and  not  available  outside 
the  ATCS  selection  process. 

The  correlation  between  SILS  Abstraction  and  the 
abstract  reasoning  test  on  the  OPM  battery  was  found  to 
be  only  .18.  Even  though  this  was  significant,  it  would 
suggest  that  the  tests  measured  different  aptitudes  or 
abilities.  A  closer  examination  of  the  OPM  test  reveals 
that  it  measures  nonverbal  reasoning,  while  the  SILS  is 
more  of  a  verbal  reasoning  measure. 

The  average  age  of  the  sample  was  relatively  young 
(26)  because  of  the  maximum  age  restriction  of  age  30  for 
initial  entry  into  the  ATCS  occupation.  Research  indi¬ 
cates  that  age  is  positively  related  to  vocabulary  scores 
and  negatively  related  to  abstraction  (Zachary,  1986). 
Zachary  also  reports  that  abstract  thinking  declines  in 
older  adults  while  vocabulary  scores  generally  increase 
during  the  formative  adult  years  and  remain  virtually 
unchanged  into  old  age.  For  purposes  of  matching  non- 
ATCS  research  subjects  to  ATCSs  on  intelligence,  cau¬ 
tion  is  advised  when  generalizing  these  findings  to  older 
subjects  because  of  the  restricted  range  of  age  of  partici¬ 
pants  in  this  study. 
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The  data  derived  from  this  study  provided  normative 
data  for  use  of  the  SILS  as  a  screening  tool  for  non-ATCS 
subjects  in  air  traffic  related  research.  Because  of  the  prior 
selection  of  this  sample  on  the  OPM  and  the  restricted 
age  range,  generalization  of  these  data  to  the  ATCS 
population,  to  selecting  older  subjects,  and  to  selecting 
subjects  from  different  cultural  groups  requires  addi¬ 
tional  research. 
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